A Spatiotemporal/Spatiotemporal-Frequency Interpretation of Apparent Motion Reversal

Temporal aliasing artifacts are common in both computer generated and natural motion sequences. One of the most striking manifestations of temporal aliasing is the apparent reversal of motion commonly referred to as the "wagon wheel effect." In this paper, we examine temporal aliasing from the standpoint of joint spatiotemporal/spatiotemporal-frequency representations. We show that apparent motion reversal can be explained using these representations, and demonstrate that a motion est imation algorithm based on such a representation (the 3-D Gabor transform) can accurately predict this illusion. 1 I n t r o d u c t i o n In temporal aliasing, components of the image sequence with high temporal frequency appear at lower frequencies due to an insufficient temporal sampling rate (commonly referred to as the frame rate). These components affect the visual quality of the sequence in two respects. The first, most noticeable at low frame rates, is a disruption in the smoothness of perceived motion (a breakdown of the apparent motion illusion underlying all motion picture display methods). Although predicting the sampling rate at which this effect wil l become visible is not tr ivial, the effect itself is intuitive the motion appears nonsmooth because too few samples are presented. The second, which may be seen at comparatively high frame rates, is a distortion of the direction and speed of the perceived motion. This distortion is common in current f i lm and video, and manifests in connection with objects that exhibit both significant high-spatial-frequency components (e.g., the spokes of a wheel) and that move wi th relatively high speed (the wheel is turning relatively rapidly). In the classical manifestation of this effect, a rotating wheel is seen to reverse its direction of rotation as the rate of rotation increases the "wagon wheel effect." This effect is not so intuitive, and to the author's knowledge the connection between aliasing and apparent motion reversal, while observed, has not been explained in the literature. The selection of an appropriate temporal sampling rate for a given application is of significant practical importance. A broad range of rates are in use, spanning from the very low rates (10 frames per second or less) used in current visual communications systems, through the moderate 24 30 frame per second rates used in animation, film, and standard television, to 60 frames per second in high definition television. Much higher sampling rates are used in scientific applications (hundreds and even thousands of frames per second), and are becoming increasingly common and economically viable as technology improves. For sequences involving high degrees of motion (particularly when detailed spatial structure must also be represented), temporal aliasing is the most important phenomenon in determining the temporal sampling rate, providing an additional motivation to understand it fully. Motion is most intuitively a spatiotemporal phenomenon. However, it has been shown that it can also be characterized in the frequency domain via Fourier analysis. [Watson and Ahumada, 1983] have used this approach to investigate the relationship between frame rate, the bandwidth of the human visual system, and the perceived smoothness of motion for a moving line stimulus. Their results clearly demonstrate the value of frequency domain analysis for understanding aspects of visual motion perception. In this paper, we consider the manner in which aliasing affects sequences at comparatively high frame rates, where motion is generally perceived as smooth, yet distortions of speed and direction may be visible. A method is demonstrated by which motion estimates consistent wi th those perceived visually are obtained, in cases both with and without aliasing, for sequences consisting of regions wi th different motions (a task that cannot be undertaken wi th conventional Fourier analysis). To do so, we utilize a generalizat ion of frequency domain motion analysis, based on a joint spatiotemporal/spatiotemporal-frequency repreNote that we distinguish here between sampling and screen update rates, the later being driven primarily by the perception of wide area flicker. Screen updates need not be unique samples, a fact reflected, e.g., in current film to video conversion practice. 1140 ROBOTICS AND PERCEPTION Figure 1: The spectrum of a static sinusoid w i th frequency and the same sinusoid moving wi th velocity sentation (the 3-D Gabor transform). 2 M o t i o n and Al iasing in the Frequency Domain The frequency domain characterization of motion has been studied for some t ime, and for certain situations is well understood. In the case of an ini t ial ly static image undergoing translat ion w i th constant velocity, motion is particularly straightforward to describe in the frequency domain. The spectrum of the sequence lies in an oblique plane, the slope of which indicates the velocity of the